Evaluating implicit judgements from Image search interactions
نویسندگان
چکیده
Click-through data derived from traditional document search has recently been closely examined with some doubts cast on its perceived usefulness as a source of absolute relevance judgements. Conceptually similar, and holding many potential uses, image search click-through data has not seen such close examination, despite vastly different properties. Addressing this we detail and report findings from a user study involving over 65 participants. Aimed at determining the reliability of image search click-through data, factors identified in previous studies in document search click-through data are examined and results compared. In addition, additional factors uniquely present in web based image search are motivated and examined.
منابع مشابه
Criteria that have an effect on users while making image relevance judgements
This paper reports the result of an exploratory user study investigating criteria that are important to users when judging relevance while performing an image search. Data was collected from 12 participants using questionnaires and screen capture recordings. Users were required to perform three image search tasks which are specific, general and abstract image search and judge relevance based on...
متن کاملEvaluating factors affecting the liveliness of Mashhad’s public spaces by relying on image perception and analysis
Aims & Backgrounds: Public spaces are very important to urban vitality. The effective role of people due to their presence, social interactions and feeling of cheerfulness, keeps the space alive. Environmental factors play a role in creating this interaction that affects space. This research was done by using image analysis, which demonstrates a wide range of information based on human percepti...
متن کاملUnbiased Evaluation of Retrieval Quality using Clickthrough Data
This paper proposes a new method for evaluating the quality of retrieval functions. Unlike traditional methods that require relevance judgements by experts or explicit user feedback, it is based entirely on clickthrough data. This is a key advantage, since clickthrough data can be collected at very low cost and without overhead for the user. Taking an approach from experiment design, the paper ...
متن کاملRelevance as process: judgements in the context of scholarly research
Introduction. This paper discusses how exploring the research process indepth and over time contributes to a fuller understanding of interactions with various representations of information. Method. A longitudinal ethnographic study explored decisions made by two informants involved in scholarly research. Relevance assessment and information seeking were observed as part of informants' own ongo...
متن کاملExploring Human Judgement of Digital Imagery
Statistical learning methods are commonly applied in content-based video and image retrieval. Such methods require a large number of examples which are usually obtained through a manual annotation process, that is human raters review images and assign semantic concept labels. The human judgement, however, cannot be regarded as the ultimate truth because of its subjectiveness and the likelihood ...
متن کامل